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SUMMARY 


This  report  covers  the  period  from  1  June  1979  to  31  May  1980,  and 
concerns  the  third  of  three  phases.  This  phase  concentrated  on  the  develop¬ 
ment  and  application  of  the  augmented  Program  Testing  Translator  (PTT) 
described  in  the  July  1979  Interim  Report  on  AFOSR  F44620-74-C-008. 

"X 

The  software  required  to  test  FORTRAN  programs,  first  with  random 
numbers  and  then  by  preselected  constructed  cases,  was  developed  during  this 
interval.  Software  modifications  were  made  to  permit  performance  measure¬ 
ments  of  the  degree  of  coverage  to  include  the  results  due  to  the  use  of  con¬ 
structed  cases.  The  automation  of  the  mix  of  random  numbers  and 
constructed  cases  was  studied  and  limitations  were  found. 


Two  new  models  for  use  during  the  development  phase  of  programming 
were  developed.  These  models  apply  to  programs  which  grow  in  size  during 
the  debugging  period. 
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Section  1 

OBJECTIVES  AND  TASK  DESCRIPTIONS 


r 


1 .  1  WORK  ACCOMPLISHED 

The  primary  effort  during  the  contract  period  encompassed  work  on  three 
tasks: 

1.  Tailor  or  expand  the  testing  programs  that  were  developed  in  the 
early  phases  of  the  contract. 

2.  Code  so  as  to  provide  valuations  of  the  program  predicates,  and 
values  of  the  artificial  program  variables  which  provide  the  data  for  the 
search  procedures. 

3.  Modify,  install,  and  test  the  tool  on  a  laboratory  computer  when  the 
scope  and  size  of  the  test  tool  are  established. 

1.2  ADDITIONAL  WORK  REQUIRED 

Continuing  studies  will  be  made  to  assess  the  practicality  of  a  fully  auto¬ 
mated  version  of  testing. 

This  additional  work  is  included  in  the  major  task: 

Test  the  tool  and  the  methodology,  using  the  several  constructs 
(connection  matrix,  status  vectors,  predicate  valuations,  and  input  and 
output  data)  through  the  implementation  on  a  "laboratory"  type  of  computer, 
such  as  the  Nanodata  QM-1. 
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Section  2 

STATUS  OF  RESEARCH  EFFORTS 


2.  1  SEGMENT  COVERAGE  BY  RANDOM  AND  CONSTRUCTED  CASES  < 

I  '  1 

I 

2.1.1  Problem  Identification 

This  particular  part  of  the  study  reached  the  point  where  the  methods  of 

j 

analysis  were  established  and  were  tested  out  acceptably.  There  are  three 

major  software  related  problems,  which  were  solved.  They  are  discussed  in  j 

the  following  three  subsections. 

;  -1 

2.  1.  1.  1  Problem  1:  Estimation  of  Number  of  Residual  Tracks 

First,  the  estimated  number  of  tracks  through  a  program  obtained  by 

using  random  numbers  as  program  drivers  was  solved.  This  problem  had 

-  j 

been  solved  in  principle,  but  implementation  of  it  heretofore  had  been  effected 

i  by  the  tedious  process  of  desk  checking  segment  usages  against  all  past 

usages.  The  software  required  to  automate  the  process  has  been  written. 

1  i 

Although  the  capability  exists  to  use  arbitrary  probability  distributions  for 
the  random  drivers,  the  usual  procedure  was  to  employ  the  same  distribution  and 
the  same  range  of  values  for  all  input  variables,  this  common  distribution 
was  a  uniform  distribution  on  a  fixed  range  (of  the  logarithm).  For  addi¬ 
tional  flexibility,  two  additions  to  the  total  APTT  program  were  added. 

First,  the  range  of  each  real  input  variable  was  made  selectable,  with  the 
distribution  over  the  logarithm  uniform  on  this  range;  second,  for  integers, 
the  selection  was  made  from  a  uniform  distribution  over  whatever  range  may 
be  selected.  A  random  (50/50)  sign  selection  was  also  made  for  the  (always 
positive)  real  variables;  for  the  integer  variables  the  range  may  be  extended 
to  negative  integers  so  that  the  device  is  not  required. 

The  selection  of  random  values  for  the  input  variables  (real  or  integer) 
provides  the  set  of  values  for  one  run.  The  procedure  employed  for  estimating 
the  number  of  tracks  that  will  be  exercised  requires  a  number  of  executions 
and  comparisons.  In  the  automatic  version,  the  track  that  accompanies  one 
^  input  data  (random)  selection  is  identified  in  terms  of  a  zero  or  one  assign¬ 

ment  to  the  arbitrarily  ordered  set  of  segments  which  comprise  the  list  of 


segments:  a  zero  for  nonusage  and  a  1  for  one  or  more  usages.  (Two  paths 
which  differ  in  their  nonzero  counts  of  the  usages  of  segments,  or  in  their 
order  of  execution,  are  considered  to  have  the  same  track). 


In  the  implementation  of  the  estimation  process,  the  above  outlined  initial 
portion  is  followed  (in  the  postprocessor)  by  a  routine  which  compares  the 
sequence  of  binary  n-tuples  (one  "ordinate"  for  each  program  segment)  in 
order  to  accomplish  two  things: 

A.  Establish  whether  a  newly  examined  track  is  the  same  as  some 
track  earlier  examined,  effected  by  comparing  the  n-tuples  ordinate  by 
ordinate  against  all  previously  taken  tracks, 

B.  Marking  the  trial  number  of  the  current  track  by  a  zero  or  1  in 
accordance  with  the  results  of  the  comparisons,  a  zero  if  an  "old"  n~tuple 
has  been  found  and  a  1  if  the  examined  track  is  new. 

The  data  for  the  estimation  procedure  consist  of  the  pattern  of  0's  and  l's 
obtained  in  the  above  comparisons.  The  primary  observable  consists  of  the 
total  trials  between  adjacent  l's.  These  spacings  between  l's  are  reported 
as  Xj,  X2,  .  .  .  ,  and  represent  the  difference  in  the  indices  representing 
trial  numbers:  Xj  is  the  separation  between  the  first  trial  number  (by  defi¬ 
nition,  the  first  trial  results  in  the  first  new  track)  and  the  trial  number 
which  produces  the  second  new  track  (usually  this  separation  is  1  because  of 
the  high  likelihood  that  a  new  data  set  will  produce  a  different  track);  X£  is 
the  separation  between  the  third  and  second  new  track,  etc. 

With  data  Xj,  X 2,  . . .  ,  Xn  obtained  by  running  the  program  over  T  trials, 
the  number  of  new  tracks  is  estimated  from  the  equation 

n 

Z  1 _  nT 

i  =  1  N-(i-l)  '  n  (1) 

NT-  S  (i-l)X. 
i=  1  1 

where  N  is  the  unknown,  X^  are  as  defined,  T  is  the  total  number  of  trials 
and  n  is  the  number  of  X.  employed. 


MCOOMWffi 


U.  OOWItAifc^ 


4 


m 


The  augmented  version  of  APPT  achieves  this  entire  process  of  com¬ 
parison  and  estimation  automatically. 

2.  1.1.2  Problem  2:  Comprehensive  Coverage 

A  certain  track  is  taken  in  response  to  any  input  data  set.  This  track  iB 
characterized  by  the  segments  that  are  exercised,  without  regard  to  their 
order  or  their  multiplicity  of  execution.  The  major  problems  in  completely 
automating  the  cover-testing  process  are  in  construction  of  the  software 
required  to  establish  the  status  of  testing,  maintain  suspense  files  on  ail 
unexercised  program  segments,  insert  augmenting  variables  corresponding 
to  predicates  which  define  the  entry  into  the  (unexercised)  computational 
segments,  search  the  input  variable  space  to  achieve  entry,  compare  the 
resulting  track  with  previously  obtained  tracks,  and  prune  the  original  tracks 
to  a  set  of  smaller  dimension  (manifested  in  the  reduction  of  the  original 
n-tuples  to  tuples  of  smaller  size). 

The  complete  list  of  tasks  required  to  develop  the  software  follows: 

A.  Identify  unexercised  branches  (at  the  end  of  the  initial  runs  with 
random  numbers). 

B.  Pick  an  unexercised  branch  and  display  the  listing  associated  with 
the  branch  (a  "back"  sort  is  required  which  identifies  the  instruction  number 
of  the  involved  predicate). 

C.  Formation  of  an  auxiliary  variable  based  on  the  nature  of  the  predi¬ 
cate.  (For  example,  if  the  test,  A<B,  is  the  predicate,  the  auxiliary  var¬ 
iable  could  be  Cj  =  B-A). 

D.  Create  a  variable  (with  requisite  modifications  to  the  object 
program). 

E.  Vary  input  variables  until  the  auxiliary  variable  is  positive. 

Rationale  for  the  variation  depends  on  the  program  variables  identified  in 
the  listing. 

F.  List  all  exercised  segments  and  compare  with  preceding  usage, 

G.  'Release"  the  variable  and  proceed  to  a  new  unexercised  branch. 

H.  In  an  extension  of  the  above  procedure,  several  auxiliary  variables 
can  be  inserted  at  one  time  and  input  data  chosen  in  some  systematic  way  (a 
search)  to  achieve  arbitrary  valuations  on  all  auxiliary  variables. 
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Not  all  of  the  software  required  for  this  problem  has  been  written  and,  in 
the  main,  the  process  of  integrating  a  display  into  a  human/computer/display 
interrogative  mode  has  not  been  accomplished.  The  progress  which  has  been 
made  toward  that  end  is  described  in  Section  2.1.2. 

2.  1.  1.3  Problem  3:  Formation  of  Execution  Sequences 

It  is  well  to  state  at  the  outset  that  only  the  outline  of  this  problem  has 
been  established.  The  following  paragraphs  describe  the  background  and  out¬ 
line  of  the  problem. 

The  use  of  tracks  as  proxies  for  execution  sequences  is  in  part  necessary 
and  in  part  expedient.  Tracks  are  necessary  because  one  usually  cannot  deter¬ 
mine  the  actual  sequence  from  a  list  of  usages:  with  several  entrances  and  several 
exits  from  a  node  and  a  different  usage  number  of  each,  there  is  usually  no  way  to 
determine  the  actual  sequence  of  the  computation  that  would  produce  the 
usage  numbers.  On  the  other  hand,  information  often  is  available  which 
would  allow  the  program  flow  to  be  determined  in  a  gross  or  general  way, 
and  that  information  heretofore  has  not  been  employed  in  our  studies.  It 
would  be  helpful  to  program  testers  to  provide  a  general  sequence  of  the  flow 
resulting  from  a  given  input  driver  set. 

To  illustrate  this,  an  example,  depicted  in  Figure  1,  shows  the  set  of  exe¬ 
cuted  segments  and  their  counts  as  solid  lines  or  arcs  between  all  nodes  which 
were  passed  during  the  first  data  set  employed  on  the  ORLA  program  described 
in  the  2nd  Interim  Report  (Reference  1).  It  will  be  noted  that  dotted  lines  are 
also  shown  emanating  from  certain  of  the  nodes  which  were  passed.  These 
are  branches  which  were  not  taken  on  the  run;  they  would  be  important  in 
coverage  testing  but  can  be  ignored  for  the  present  discussion.  The  flow  of 
the  computations  can  be  determined  unambiguously  only  in  the  cases  where  a 
single  execution  is  performed  on  a  segment  and  no  other  segment  parallels 
the  segment.  For  example,  there  is  such  a  segment  joining  nodes  355  and 
263  in  the  central  lower  one-third  of  the  chart.  This  and  others  are  highlighted 
in  Figure  2,  where  they  may  be  easier  to  locate. 

The  general  flow  can  be  formed  from  the  unambiguous  segments  which 
show  a  usage  of  one.  In  one  case,  there  are  (at  node  226)  two  segments,  both 
with  a  count  of  1,  shown  exiting  a  node.  But  this  particular  ambiguous  case 
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is  easily  resolvable  (i-.  e.  ,  precedence  determined)  because  the  branch  along 
segment  13,  joining  nodes  226  and  483,  joins  to  the  exit  (END),  and  so  cannot 
precede  the  segment  joining  nodes  226  and  235.  This  suggests  an  interesting 
problem  of  which  the  preceding  example  is  the  most  trivial:  given  a  set  of 
nodes  and  their  counts,  determine  under  what  conditions  the  actual  flow  can 
be  determined.  This  "academic"  problem  will  not  be  pursued  in  this  study. 

The  application  of  the  simple  rule  which  establishes  the  one-time  used 
segments  (a  "footprint,"  or  better,  a  "one-print")  permits  a  linking  of  certain 
segments  to  form  contiguous  blocks  of  the  program,  the  General  Flow  of  the 
title  of  Figure  2. 


Such  linkings  are  shown  in  Figure  2,  where  the  defined  flow  consists  of 
the  following: 

Block  1:  Segments  1,  113,4,6,112,8,102,11,87,101 

Block  2:  Segments  96,  18,86 

Block  3:  Segment  26 

Block  4:  Segments  30,  32,  33, 45,  36 

Block  5;  Segments  41,42,  13,  15,  17  (END) 

Even  the  undefined  flow  can  be  combined  to  form  pseudo  segments  if  there 
are  no  dotted  lines:  thus,  the  series /parallel  segments  20,21,84,22,23,24, 
and  25,  which  are  between  nodes  319  and  355,  can  be  treated  as  a  single 
pseudo  segment  with  a  usage  of  150,  the  entry  and  exit  counts  at  the  two 
joined  nodes.  In  addition  to  these  pseudo  segments,  another  type  of  merging 
is  possible  in  certain  areas.  For  example,  some  of  the  segments  from 
Block  4  of  the  above  list  can  be  joined  with  the  segment  of  Block  3  to  form  a 
superblock.  Since  all  possible  paths  to  and  from  nodes  263  and  273  have  been 
exercised,  these  can  be  eliminated  from  further  consideration,  permitting 
formation  of  a  pseudo  segment  with  which  to  join  segment  30  to  segment  26. 
Also,  since  node  291  has  all  exits  exercised,  it  too  can  join  to  form  a  larger 
block  (26,  pseudo  segment,  30,  and  32).  Because  node  292  has  a  dotted  line 
out  of  it,  there  is  no  further  merging  possible  between  the  two  blocks. 

Even  though  the  remainder  of  the  program  flow  is  undefined,  there  are 
many  points  which  are  internal  to  the  undefined  blocks  where  reduction  is 
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possible.  A  trivial  example  is  the  pair  of  parallel  paths  91  and  99  between 
nodes  191  and  209,  which  can  be  merged  into  a  two-use  segment;  more 
interesting  cases  can  be  identified  in  the  lower  left  portion  of  Figure  2.  Thus, 
between  nodes  401  and  415  are  segments  53,  68,  54,  55,  57,  56,  and  58,  all 
of  which  can  be  merged  to  a  118-use  pseudo  segment. 

Figure  3  shows  a  considerably  pruned  version  of  the  flow  diagram.  As 
with  the  preceding,  it  is  developed  from  the  one-prints  and  more  is  required 
to  establish  the  sequence.  For  example,  segment  42  appears  to  follow 
(dynamically)  41,  but  there  is  no  reason  to  think  a  priori  or  in  a  local  context 
that  it  actually  does.  In  a  global  context,  however,  it  is  known  that  segment 
42  is  the  later  exit  from  node  385,  because  42  joins  to  226  and  from  there  out 
to  the  END. 

2.1.2  Software  Implementations 

2.  1.2.1  The  Program  Testing  Translator 

The  PTT  and  its  augmentation,  APTT,  are  the  primary  software  tools 
supporting  the  Software  Quality  Metrics  effort.  Use  of  the  PTT  tool  in  earlier 
phases  of  the  study  allowed  individual  subroutine  driving  by  employing  random 
variables  for  each  subroutine  parameter.  In  applications,  data  was  used  to 
drive  the  subroutine  repeatedly;  this  was  followed  by  collecting  statistics  on 
execution  counts  and  branchings.  Then  a  postprocessing  step  was  performed 
which  presented,  in  tabular  form,  a  list  of  segments  and  the  corresponding 
segment  usage  statistics.  This  process  in  the  early  phases  required  a  manual 
connecting  of  segments  into  possibly  larger  segments.  Noticeably  absent 
from  the  early  version  of  the  PTT  tool  were  branches  implied  in  DO- LOOP 
termination,  and  in  END=  and  ERR=  branches  in  file  operations. 

To  alleviate  some  of  the  shortcomings,  the  PTT  was  modified  in  several 
respects.  First,  it  was  altered  to  operate  on  the  entire  program  instead  of 
subroutines.  This  allowed  analysis  of  all  FORTRAN  modules  in  a  program 
to  occur  at  one  time.  The  PTT  was  also  upgraded  to  recognize  the  implicit 
DO,  END,  and  ERR  branches.  Along  with  these  changes,  the  segment  anal¬ 
ysis  was  completely  redesigned  to  allow  composition  of  complete  dynamic 
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segments  automatically  and,  to  a  small  degree,  to  pinpoint  unreachable 
portions  of  code.  This  redesign  also  caused  a  more  efficient  allocation  of 
segment  monitors. 

2.  1.2.2  Augmentations  of  PTT 

Since  the  redesign,  the  three  problems  discussed  in  Section  2.1.1  have 
arisen  and  these  required  further  modifications  to  the  PTT.  The  first  was 
to  obtain,  automatically,  an  estimate  of  the  number  of  residual  tracks  remain¬ 
ing  to  be  found.  The  solution  involved  addition  of  an  input  description  record 
by  the  user  to  control  ranges  on  the  random  variables  used  as  drivers,  and  to 
get  a  new  set  of  input  variables  upon  calling  of  the  input  routine.  The  gathered 
execution  statistics  numbered  each  case  run  and  determined,  with  use  of  the 
postprocessor,  the  X.,  required  to  calculate  the  residual  number  of  tracks. 

The  second  problem  required  software  assistance  in  the  selection  of  test 
data,  first  by  identifying  the  unexercised  branches  and,  second,  to  allow  the 
user  to  add  one  or  more  new  temporary  auxiliary  variables  to  be  monitored. 
Cases  would  then  be  run  and  the  values  of  the  input  variables  varied  until 
a  positive  valuation  of  the  temporary  auxiliary  variables  occurred.  The 
choice  of  the  auxiliary  variables  and  the  equations  which  define  them  are  left 
to  the  user  in  this  version  of  PTT.  The  PTT  assists  the  user  by  compiling 
so  as  to  incorporate  the  auxiliary  variables,  generating  a  monitoring  code, 
and  by  updating  the  list  of  unexercised  branches. 

The  third  problem,  discussed  in  Section  2.  1.1.3,  was  to  assist  the  user 
with  a  method  of  determining  the  program  flow.  This  problem  presented  a 
level  of  difficulty  such  that  no  immediate  assistance  from  the  PTT  per  se 
was  deemed  possible. 

2. 1.2.3  Examples  of  Applications 

To  illustrate  each  of  the  first  two  problems  and  the  PTT  solution,  the 
matrix  triangularization  example  will  be  reexamined.  This  example  is 
extensively  discussed  in  the  2nd  Annual  Report  (Reference  1),  where  directed 
graphs  of  the  potential  program  flow  and  examples  of  the  coverage  by  random 
numbers  and  constructed  cases  were  given.  Listings  of  the  MAIN  and 
TRIANGULARIZATION  subroutine  comprise  Figure  4. 
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Figure  4.  Listing  of  an  Example  Program  (Page  1  of  2) 
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Figure  4.  Listing  of  an  Example  Program  (Page  2  of  2) 


Appendix  A  contains  tables  and  reports  of  the  PTT  output  for  three 
separate  test  runs.  The  reports  show  the  testing  coverage  provided  by  using 
the  user-described  input  routine  INROUT.  INROUT  returns  a  new  set  of 
random  distributed  data  values  for  the  input  matrix  A.  The  data  values  are 
uniformly  distributed  over  the  logarithm  in  the  range  -2  to  1.  The  sign  of 
the  individual  data  items  is  also  selected  randomly. 

The  first  three  cases  of  test  run  No.  1  (see  Pages  A-l  to  A-3  in  Appendix 
A)  show  coverage  of  code  for  the  MAIN  program  as  100%  in  the  column  marked 
Summary.  Subroutine  TRIANG  gets  a  summary  coverage  of  86.96%.  The 
remaining  segments  to  be  tested  are  numbers  3,  12,  and  16,  as  seen  in  the 
segment  reference  report  (Page  A-2  of  Appendix  A).  The  segment  reference 
tables  are  used  to  relate  the  segment  numbers  and  their  corresponding 
program  statement  numbers  together.  As  an  example,  it  is  seen  that  segment 
3  contains  lines  34,  35,  36,  and  37  in  subroutine  TRIANG  (see  Figure  4). 

These  lines  correspond  to: 

IF(A(K,K).  EQ.  0)34  IP(N)  =  035 

CONTINUE  -  K  =  K+l36  IF  (K.  LE.N)37  Loop 

(DO-loop  termination  includes  an  implied  conditional  branch) 

Following  the  summary  reports  and  the  segment  reference  tables,  the 
trial  statistics  appear  on  Page  A-3,  for  example.  These  are  the  X.  that  are 
needed  to  calculate  the  estimate  of  the  number  of  remaining  tracks.  (Actually, 
more  than  three  cases  are  required  for  the  estimation  and  the  three  entries 
on  Page  A-3  form  only  a  part  of  the  data  used.) 

Supplied  as  part  of  the  testing  package  is  a  program  that  interacts  with 
the  user  and  calculates  the  difference  of  the  two  sides  of  equation  (1)  in 
Paragraph  2.  1.  1.  1  based  on  trial  solutions  supplied  by  the  user. 

To  explain  how  the  are  formed  the  formation  of  Xj  and  X ^  will  be 
considered.  Case  1  of  run  1  (see  Page  A-l)  shows  the  number  of  times  each 
segment  of  MAIN  and  TRIANG  were  executed.  Since  this  is  the  first  test 
case,  the  first  unique  track  is  automatically  formed.  Case  2  of  run  1  for 
TRIANG  (Page  A-l)  shows  the  same  segments  being  executed  (the  number  of 


Mcpowwtu 


16 


executions  of  each  segment  listed  happen  to  be  the  same,  but  this  is  irrelevant; 
the  comparison  is  made  on  the  basis  of  whether  or  not  the  segment  was  execu¬ 
ted,  not  on  how  many  times)  as  in  case  1,  run  1.  The  MAIN  routine  shows  a 
difference  in  execution.  Therefore  case  1  and  case  2  are  different,  so  we  form 
Xj  =  1.  This  means  that  one  case  occurred  since  the  last  unique  track.  If  we 
compare  case  3  of  run  1  against  cases  1  and  2  we  also  find  a  difference  in  the 
MAIN  routine  (see  segment  3  execution  counts).  This  gives  us  our  third 
unique  track.  Hence,  X^  =  1,  also,  since  only  1  case  occurred  since  the  last 
unique  track. 

Continuing  in  this  fashion,  by  comparing  cases  1  through  9  (in  Appendix 
A),  in  order,  we  find  unique  tracks  for  cases  1,  2,  3,  4,  and  8.  (A  summary 
of  the  nine  cases  is  found  on  Page  A-9.)  The  Boolean  tokens  associated  with 
the  sequence  are  shown  below: 

•v^CASE^  123456789 
NEW/  111100010 

old 

X=1  X  =1  X-=l  X  =4 
12  3  4 

By  using  the  estimation  equation,  it  was  determined  that  there  existed  9.  1 
new  tracks  to  be  found. 

Appendix  B  contains  reference  tables  of  the  PTT  output  for  a  constructed 
case.  The  constructed  case  shows  the  use  of  monitor  variables  (Page  B-2). 

For  constructed  cases,  the  user  is  required  to  supply  input  data  to  the  pro¬ 
gram,  and  to  supply  the  monitor  variables.  It  is  seen  that  the  user- supplied 
input  is  in  the  DATA  statement  in  the  MAIN  program.  Subroutine  TRIANG 
shows  the  use  of  monitors  inserted  into  the  program  of  branch  points. 

By  analyzing  the  unexercised  segments,  3,  12,  and  16  of  the  three  test 
runs  of  Appendix  A,  where  they  are  marked  by  asterisks  in  all  three  of 
the  segment  reference  tables  (Pages  A-2,  A-5,  A-8),  it  can  be  determined 


from  the  listing  that  the  variable  T  holds  the  key  to  exercising  these  segments. 
Further  examination  suggests  that  if  A[3,3]  is  equal  to  zero  then  segment  3 
will  be  exercised. 

Segment  12  requires  variable  T  to  be  zero.  For  this  to  be  true,  A[l,  l] 
could  be  equal  to  zero  or  |A[3,  2]j could  be  greater  than|A[2,  2]|  and  A[l,2]  must 
be  equal  to  zero. 

Segment  16  also  requires  variable  T  to  be  equal  to  zero.  This  condition 
will  result  if  A[l,  l]  is  not  zero  and  A[l,2]  is  equal  to  zero. 

These  findings  determined  the  initial  values  of  A  for  the  DATA  statement. 
By  observing  the  segment  reference  for  subroutine  TRIANG,  we  find  that  seg¬ 
ments  3,  12,  and  16  have  been  executed  and  the  test  coverage  is  complete. 

2.2  ERROR- DETECTION  MODELS 

2.2.1  Summary 

Two  variations  of  the  Jelinski-Moranda  model  were  developed  for  esti¬ 
mation  during  program  development.  The  first  permits  estimation  of  the 
error  content  of  the  completed  software  package  using  data  which  is  taken 
on  only  portions  of  the  package.  That  model  is  applicable  when  the  eventual 
size  of  the  program  is  known  at  the  outset. 

The  second  model  permits  a  similar  analysis  during  the  development  of 
any  software  package  which  is  homogeneous  with  respect  to  its  complexity 
(error  making  and  finding). 

These  models  should  assist  analysts  in  an  early  determination  of  error 
content.  They  should  also  eliminate  the  present  practice  of  applying  models 
to  the  wrong  regime  (decreasing  failure  rate  models  applied  to  growing-in¬ 
size  software). 

2.2.2  Introduction 

In  normal  usage  of  the  Jelinski-Moranda  model,  the  software  package 
under  test  is  assumed  to  be  of  fixed  size  with  a  fixed  number  of  incipient 

MCOOWWffli. 


17 


errors.  The  size  of  the  package  does  not  appear  explicitly  in  the  model  as  a 
parameter,  and  its  effect  is  only  indirectly  realized  by  the  way  it  affects  the 
number  of  incipient  errors  which  exist  at  the  start  of  testing  (there  is  a  direct 
relation  between  instruction  count  and  error  count). 

That  model  could  not  be  employed  legitimately  on  software  packages 
which  were  incomplete.  Several  workers  attempted  to  fit  the  model  to  an 
initial  period  of  time  when  its  error  rate  was,  indeed,  increasing,  due  to  the 
growing  size,  and  they  met  with  no  success.  (As  a  matter  of  fact,  the  only 
models  which  produced  reasonable  estimates  when  applied  during  this  regime, 
were  the  increasing  failure  rate  models). 

It  would  be  helpful  if,  at  the  outset,  an  estimate  could  be  obtained  of  the 
total  error  count  which  will  be  realized  in  test  and  usage  of  a  package. 

Recent  work  by  IBM  (Reference  2)  has  prompted  a  reexamination  of 
the  original  Jelinski-Moranda  model  for  the  purpose  of  incorporating  the 
(changing)  program  size.  This  turns  out  to  be  very  easily  effected  if  good 
record  keeping  can  be  maintained  during  program  development  so  that  the 
size  of  the  package  is  recorded  as  a  function  of  some  convenient  timing 
metric  (CPU  or  calendar).  Following  is  a  description  of  the  analysis. 

The  original  model  is  depicted  in  Figure  5,  where  the  two  parameters 
are  shown  in  Figure  5(b),  and  a  typical  realization  of  the  error-finding  proc¬ 
ess  is  shown  in  Figure  5(a).  N  is  the  initial  error  content  (of  a  completed 
program)  and  4>  is  the  contribution  to  the  error  rate  due  to  a  single  error. 

While  the  meaning  of  <p  is  maintained  in  the  two  models,  the  meaning  of 
"initial  error  content"  needs  to  be  clarified.  This  is  done  below  in  the 
description  of  Model  1,  where,  in  effect,  N  maintains  its  meaning  as  the 
number  of  errors  in  a  completed  package.  In  the  second  model,  a  fixed 
error  rate  per  instruction  is  assumed,  and  growth  of  the  package  is  meas¬ 
ured  by  the  count  of  instructions  (under  test)  versus  time. 
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(a)  Typical  Realization 


Figure  6.  Purification  Process  and  its  Realization 

2.2.2.  1  Model  1 

Let  S  (t)  denote  the  fraction  of  the  total  number  of  statements  which  a 
complete  program  will  have.  The  metric  t  is  measured  in  terms  either  of  the 
accumulated  CPU  time,  or  of  the  amount  of  calendar  time,  which  has  been 
used  for  testing  the  package. 

The  simplest  way  of  introducing  the  effect  is  to  use  S(t)  as  a  "modula¬ 
tion"  of  the  error  detection  rate  Z(t)  of  the  original  model.  In  the  notation 
formerly  employed,  this  combined  or  modulated  rate,  denoted  W(t),  is: 

W(t)  =  S(t)  Z(t) 

=  S(t)  f (N-i+ 1  )♦]  for  T!_  j  <  t  <  T!  (2) 

and  T'l,  T£,  T^ . denote  the  times  of  detection  for  the  errors.  (Primes 

are  employed  on  T's  to  distinguish  them  from  the  times  of  the  original  proc¬ 
ess.)  The  effect  of  S(t)  on  the  T!  should  be  made  clear  at  the  outset.  When 


f 


S(t),  the  fraction  of  the  total  count,  increases,  the  composite  error  rate  will 
generally  increase,  as  will  the  liability  for  error  for  the  "modulated"  process. 
For  this  reason,  the  times  Tj  for  the  composite  process,  W(t),  will  be  differ¬ 
ent  from  the  T\  of  the  Z(t)  process.  Since  N  in  the  original  model  represented 
the  total  error  content  of  a  complete  software  package,  a  proper  correspond¬ 
ence  which  preserves  the  meaning  is  that  N  is  the  error  content  at  a  time 
corresponding  to  the  completion  of  the  software  package,  S(t)  =  1.00.  This 
necessarily  presumes  that  the  size  of  the  package  which  will  be  developed 
is  known  at  the  outset.  (SQ(t)  would  represent  the  fraction  of  the  total  which 
is  accomplished  at  time  t. )  This  may  or  may  not  be  a  serious  barrier.  Some 
modules  can  be  sized  at  the  outset,  but  large  complex  programs  may  not  be. 

An  alternative  to  this  is  offered  subsequently  in  Model  2. 

For  the  present  model,  SQ(t)  is  a  nondecreasing  function  which  starts  at 

zero  at  T^  and  achieves  its  maximum  value  at  some  unknown-at-the-outset 

time,  T'  . 
c 


Thus,  0  <  SQ(t)  <  1,  with  So(TQ)  =  0  and  Sq(Tc)  =  1. 

While  SQ(t)  is,  in  the  large  sense,  random,  the  records  of  progress  will 
permit  specific  values  of  SQ(t)  to  be  determined  and  the  randomness  is  of  no 
concern.  In  particular,  it  is  necessary  that  SQ(t)  can  be  determined  at  the 
epoch  times  Tj,  T!,» . . .  ,  T^  at  which  the  errors  are  detected. 

When  the  completion  time,  T^.,  is  reached  and  for  times  thereafter,  the 
software  package  is  complete  (Sq(T^)=1)  and,  formally,  the  density  given  in 
Equation  (1)  is  the  same  as  that  given  in  the  original  paper  (Reference  3). 

It  has  been  mentioned  earlier  that  the  time  pattern  of  errors  will  be  dif¬ 
ferent  for  the  "modulated"  process,  and  it  is  interesting  to  see  just  what 
would  happen  if  Sq(T^),  or  for  short,  Sq(0),  were  0.  10  (10%  of  the  package  is 
initially  available  for  test),  and  it  did  not  increase  beyond  that  for  a  long 
period  of  testing.  The  time  pattern  of  errors  T'j,  T!,, . . . ,  T^  which  would 
occur,  would  have  associated  separations  X'^  =  T'j  -  T^,  X'j  =  T£  -  T'^,  ..., 
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Because  So(0)  =  0.  10,  the  composite  detection  rate  for  the  first  error 
would  be  (0.  10)  N$,  that  is,  10%  of  the  original  error-detection  rate.  This 
means  that  the  first  detection  time  Tj,  would  (on  the  average)  be  10  times 
as  long  as  the  time  for  the  corresponding  error  of  the  unmodulated  process. 
The  second  error  would  have  the  same  property  (on  the  average),  and  so 
forth.  The  implications  of  this  fact  can  be  seen  from  the  following.  The 
likelihood  function  would  be 

L(X'j,  X^,  ....  X^)  = 

II  So(0)*[N-(i-l)]  exp  {  -[So(0)*(N-i+l)X!]}. 
i=  1 


The  likelihood  equations  obtained  by  differentiating  the  logarithm  of  the 
likelihood  with  respect  to  N  and  $  are: 


t  fMby  -  so<°»  *  £  xi  ■ 0 


and 


n 


So(0)  £  [N-(i-  1)]X!  =  0 


(3a) 


(3b) 


As  noted  above,  the  observables  X!  would  be  (about  or  on  average) 

10  times  as  large  as  before.  Thus,  from  Equation  (3b),  the  solution  <t>  will 
be  (on  average)  the  same  as  its  value  for  the  unmodulated  process,  or  for  the 
completed  software  package. 


Using  the  solved-for  value  of  $  in  Equation  (3a)  and  the  fact  that  Sq(0)Xj 
in  the  new  process  is  the  same  as  X.  in  the  original  process,  it  is  seen  that 
the  solution  N  is  also  the  same  as  before. 


21 


The  analysis  then  shows  that  if  it  is  known  that  a  package  under  test 
represents  (in  aLl  respects)  a  certain  percentage  of  the  total,  then  the  total 
eventual  error  content  can  be  estimated  by  using  these  slightly  modified  likeli¬ 
hood  equations. 

The  result  is  encouraging  for  the  outlook  for  success  in  the  following 
simple  generalization  of  the  above  example.  In  this  generalization,  the  SQ(t) 
modulating  function  is  constrained  to  be  constant  during  each  test  interval. 
Using  essentially  the  same  notation  as  before,  the  likelihood  equations  for 
the  generalized  modulated  process  are 


N-i+1  -  ♦  £Si-l 


X)  =  0 

1 


(4a) 


and 


4-^S.  , (N-i+l)X!  =  0 
<j>  i=l  i-l  i 


(4b) 


where  is  the  percentage  completion  achieved  prior  to  the  start  of  the  i£h 
interval. 

Solutions  for  the  parameters  can  be  carried  out  as  indicated  above  in 
the  example. 

The  mean-time -to -next  error  MTTF  (n+1  st  in  the  present  context)  can 
be  estimated  by  evaluating  the  rate  at  time  T'  and  taking  the  reciprocal  of  it. 
In  the  present  case  (using  a  subscript  on  the  left  side  to  correspond  to  the 
model  number): 

MTTF  1 

S(T^)N-n)$ 

A  ** 

where  N  and  $  are  solutions  to  the  Maximum  Likelihood  Equations  (MLE's). 
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2. 2. 2. 2  Model  2 


Let  Ep  denote  a  characteristic  rate  of  error-making  for  the  programmer 
(or  programmer  team)  and  the  program  type.  This  rate  will  be  estimated 
by  application  of  the  model  described  subsequently,  but  there  are  some  useful 
facts  concerning  this  parameter. 


In  1975,  it  was  observed  (Reference  4)  that  there  appears  to  be  a 
".  .  .  'universal*  coding  -  error  rate.  .  which  has  a  value  of  about  2  errors 
per  100  instructions  (of  the  language  in  which  the  program  has  been  written). 
This  observation  was  based  primarily  on  the  data  (now  famous)  provided  by 
F.  Akiyama,  but  also  on  earlier  observations  made  by  B.  J.  Hatter,  et.  al. 
Subsequently,  the  validity  of  this  "thermodynamically  stable"  parameter  has 
been  reinforced  by  several  other  studies. 


The  interesting  feature  of  some  of  this  later  data  (Reference  5)  is  that 
the  error  rate  of  two  per  hundred  was  observed  on  programs  which  had 
completed  their  development  and  integration  phases;  they  were  under  test 
before  the  relevant  error  counting  was  initiated.  This  is  surprising  since  the 
coding  error  rate  is  thought  of  as  being  similar  to  a  typist's  miskeying,  and 
should  be  purifiable  by  edit  routines  and  by  code  checking  due  to  early  mis- 
starts  of  the  program. 

These  features  of  an  hypothesized  entity  are  fortunately  not  used  in  the 
following  analysis. 

The  error  rate  at  any  point  in  the  development  of  a  program  whose  cur¬ 
rent  instruction  count  is  G(t)  is  assumed  to  be  proportional  to  the  current 
error  content 

V(t)  =  *  [G(t).Ep  -  n(t)]  (5) 

where  n(t)  is  the  accumulated  number  of  error  corrections,  and  is  the 
per  instruction  error  rate. 
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As  before,  if  G(t)  can  only  change  at  error-discovery  epochs,  Tj, 

.  .  .  ,  T  ,  and,  if  n(t)  also  has  this  feature,  then  the  rate  has  the  form 

V(t)  =«[G._1  Ep  -  (i-1)]  for  T.  j  <  t  <T.  (6) 

where  G._j  =  G(T^_j),  and  n(t)  is  i-1  for  the  interval  starting  at  T\ 

Since  G(t)  is  a  function  or  process  which  takes  place  without  any  apparent 
dependence  on  the  error-finding  process  (except  that  the  error  epochs  are 
assumed  to  be  the  points  of  entry  of  new  code)  it  is  reasonable  to  assume  that 
the  random  time  separations  between  errors  (X.,  X,,  ...,  X  )  are  statis- 
tically  independent. 


Under  these  conditions,  the  constant  rate  implies  an  exponential 
distribution  for  the  X.,  and  the  likelihood  function  for  n  errors  is: 


L(Xlf  X2,  ...,  Xn)  = 


n 


n  *  fGi-  l  Ed  -  ■ 1  >J  ^p  {  -dX  [G  E  -  (i-  1 )]  } 

i=  l  1  P  1  1-1  P 


(7) 


The  MLE's  obtained  by  differentiating  the  logarithm  of  the  likelihood 


function  with  respect  to  <b  and  E  are; 

P 


n 


Gi-1 


n 


(8a) 


T-L  fGi_i  Ep-(i-l)]  X.  =  0 


(8b) 


The  MLE's  are  solved  as  before;  Equation  (8b)  can  be  algebraically 
solved  for<>;  this  is  substituted  in  Equation  (8a),  and  the  resulting  key  equa¬ 
tion  is  solved  for  E  by  trial  and  error. 


It  is  recalled  that  the  desired  performance  parameter  is  E^,  which  can 
then  be  used  with  either  the  current  (known)  or  peojected  (estimated)  instruc¬ 
tion  count  to  determine  the  total  error  content. 

Estimates  of  the  MTTF  at  any  time  can  be  obtained  by  the  formula 

mttf,  =  ; - r - -  (9) 

*t°nVnl 

2.2.3  Conclusions 

The  two  models  presented  in  the  analysis  are  both  very  tractible 
analytically. 

Model  1  would  be  of  use  for  those  programs  whose  eventual  size  is  known 
at  the  outset.  It  requires  that  a  record  of  the  times  of  error  occurrences  be 
maintained  as  well  as  a  record  of  the  percentage  of  completion  at  each  of  the 
error-detection  times.  It  provides,  at  any  stage  of  testing,  an  estimate  of 
the  error  content  of  the  untested  complete  package. 

Model  2  applies  to  any  developing  software  package  which  is  homogenous 
with  respect  to  the  complexity  of  programming  and  with  respect  to  the  talents 
of  the  programmers.  The  important  property  is  that  E^,  the  error-making 
rate  (or  error-finding  rate),  must  be  a  constant  across  the  entire  software 
package.  In  case  of  inhomogeneity  separate  analyses  are  advised. 

2.2.4  Glossary 

Terms  and  symbols  used  in  the  preceding  sections  are  identified  as 
follows: 

S  (t)  A  "modulation  function"  which  ranges  from 

0  5  SQ(t)  s  1.0  and  is  nondecreasing.  It  represents 
the  fraction  of  the  code  completed  up  to  time  t.  It 
is  a  given  for  the  problem. 

Z(t)  The  Jelinski-Moranda  detection  or  purification 

process . 

W(t)  The  product  of  S(t)  and  Z(t).  It  represents  the  error¬ 

making  or  error-detecting  rate  versus  time  for 
Model  1. 


N 

* 


n 

S. 

1 


MTTFi 

G(t) 


n(t) 

V(t) 

L(XrX2, 


The  number  of  errors  in  the  completely  coded  soft¬ 
ware  package.  This  is  estimated  from  data. 

The  contribution  of  one  error  to  the  detection  (failure) 

rate.  This  is  estimated  from  data. 

th 

The  time  at  which  the  i—  error  is  found,  measured  in 

any  convenient  timing  metric.  An  observable. 

s  t 

The  separation  between  the  i —  and  the  i-l“  error. 

An  observable. 

The  cumulative  number  of  errors  found  in  testing  up 

to  time  T 1 . 

n 

Is  the  percentage  of  completion  during  the  (i+1)  st 
interval.  This  is  provided  as  exogenous  data. 

The  estimated  meantime  to  error  obtained  by  using 
Model  i  (i  =  1,2). 

The  nondecreasing  function  representing  the  total 
instruction  count  of  the  package  at  time  t.  This  is  a 
given  for  the  problem. 

The  error  making  rate  for  a  given  program-program¬ 
mer  mix.  It  is  estimated  from  data. 

The  number  of  errors  found  during  test  up  to  time  t. 
An  observable. 

A  stochastic  process  representing  error-making  or 
error -detecting  rate  versus  time. 

Xn)  The  generic  representation  for  the  likelihood  function. 
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Section  3 
PUBLICATIONS 


3.  I  STATUS  OF  PREVIOUSLY  REPORTED  PUBLICATIONS 

1.  P.  B.  Moranda,  "Event-Altered  Rate  Models  for  General  Reliability 
Analysis,  "  published  in  the  IEEE  Transactions  on  Reliability  (December 
1979). 

2.  P.  B.  Moranda,  "Probability- Based  Models  for  the  Failures  During 
Burn-in  Phase,  "  under  review  for  publication  in  the  Journal  of  the  Operations 
Research  Society  of  America. 

3.  P.  B.  Moranda,  "Comments  on:  Competing  Software  Reliability 
Models,"  to  appear  in  IEE  Transactions  on  Software  Engineering,  September 
1980. 

4.  P.  B.  Moranda,  "Comments  on:  How  to  Measure  Software  Reli¬ 
ability  and  How  Not  To,  "  incorporated  in  a  referee  report  to  the  author  on 
material  submitted  to  a  workshop  on  quantitative  software  models,  Kiamesha 
Lake,  New  York,  9-11  October  197  9. 

3.2  NEW  WORK  IN  PROGRESS  OR  PUBLISHED 

P.  B.  Moranda,  "Error  Detection  Models  for  Application  During 
Program  Development,"  submitted  to  IEEE  Transactions  on  Reliability, 
under  revision  for  resubmittal. 

3.  3  COMMENTS  AND  REVIEWS 

P.  B.  Moranda,  Review  of  "Statistical  Methods  in  Computer  Per¬ 
formance  Analysis,"  in  Current  Trends  in  Programming  Methodology,  for 
Computing  Reviews,  February  1980. 
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Section  4 


KEY  PERSONNEL 

ZYGMUNT  JELINSKI  -  Branch  Chief,  Computer  Sciences 
BS,  Economics  and  Statistics,  University  of  London,  1952. 

MBA,  Business  Administration,  Metropolitan  College,  England,  1953. 

MS,  Mathematical  Statistics,  University  of  London,  1954. 

Mr.  Jelinski  has  been  managing  computer  research  and  development 
programs  for  23  years.  As  Branch  Chief,  Computer  Sciences,  at  MDAC-HB, 
he  currently  directs  research  in  software  reliability,  modeling,  validation 
and  verification,  language  processing,  emulation,  and  simulation.  Software 
tools  developed  and/or  maintained  under  his  direction  include  the  Program 
Evaluator  and  Tester  (PET),  SUMC  Meta-assembler,  Compiler  Writing 
System,  OPAL  Development  Tool  (OPALDET),  CAMIL,  and  the  Generalized 
Language  Processor  (METRAN).  He  was  study  manager  of  a  contract  to 
develop  methodology  for  effective  test  case  selection  (a  research  contract  for 
the  National  Bureau  of  Standards),  and  he  directed  a  study  for  NASA  on 
methodology  for  reliable  software.  Another  program  under  his  direction  was 
one  in  which  software  validation  methodology  was  developed  and  applied  to  a 
tactical  software  system,  resulting  in  accurate  prediction  of  software  mal¬ 
functions.  He  also  directed  the  Software  Reliability  Study  sponsored  by  the 
Air  Force  Office  of  Scientific  Research.  This  study  involved  research  into 
the  development  and  evaluation  of  mathematical  models  representing  the  pat¬ 
tern  of  software  malfunctions.  At  Rockwell  International,  Mr.  Jelinski  was 
Chief  of  Systems  Programming  Technology.  He  directed  design  and  retrieval 
systems  and  engineering  design  aids.  Earlier,  he  managed  all  programming 
for  the  RECOMP  II  and  III  computers.  At  Philco  Corporation,  he  was  man¬ 
ager  of  programming  in  support  of  Philco  2000  computer  marketing,  and  at 
RCA  he  was  manager  of  applications  for  RCA  4310  communication  computers. 

Mr.  Jelinski's  publications  include  the  following: 

HOLDET  -  Higher  Order  Language  Development  and  Evaluation  Tool 
(coauthor),  MDAC  Paper  WD  2769,  presented  to  Computers  in  Aerospace 
Conference,  Los  Angeles,  November  1977  (AIAA,  NASA,  IEEE,  ACM). 
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An  Approach  to  Solution  of  Problems  with  Support  Software  as 
Deliverables,  MDAC  Paper  WD  27  59,  presented  to  Defense  Systems  Manage¬ 
ment  Review,  Ft.  Belvoir,  Virginia,  March  1978. 

Recent  Software  Development  Techniques  in  the  United  States,  MDAC 
Paper  WD  2706,  presented  to  Polish  Academy  of  Sciences,  Warsaw,  Sep¬ 
tember  1976. 

Software  Reliability  Predictions,  with  Dr.  P.  B.  Moranda,  MDAC 
Paper  WD  2482,  presented  to  the  Federation  for  Automatic  Control,  Boston, 
and  published  in  its  proceedings,  August  1975. 

Can  Statistics  be  Applied  to  Software  -  Historical  Perspective,  MDAC 
Paper  WD  2531,  presented  to  the  Computer  Science  and  Statistics  8th  Annual 
Symposium  on  the  Interface,  Los  Angeles,  February  1975. 

Applications  of  a  Probability- Based  Model  to  a  Code-Reading  Experi¬ 
ment,  with  Dr.  P.  B.  Moranda,  MDAC  Paper  WD  2067,  presented  to  the 
Symposium  on  Software  Reliability  Sponsored  by  IEEE,  New  York,  and  pub¬ 
lished  in  its  proceedings,  April-May  1973. 

Generalized  Events -Oriented  Simulation  System  (GESS)  -  A  Performance 
Evaluation  Tool,  with  Dr.  G.  S.  Chung,  MDAC  Paper  WD  2033,  presented  to 
Computer  Performance  Evaluation  Users  Group  sponsored  by  National  Bureau 
of  Standards,  Washington,  D.  C.  ,  published  in  proceedings,  October  1972. 

Software  Reliability  Research  (with  Dr.  P.  B.  Moranda),  MDAC  Paper 
WD  1808,  presented  to  the  Conference  on  Statistical  Methods  for  Evaluation 
of  Computer  Systems  Performance,  Providence,  Rhode  Island,  and  published 
in  its  proceedings,  November  1971. 
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PAUL  B.  MORANDA  -  TECHNICAL  ADVISOR 
AB,  Chemistry,  Fresno  State  College,  1942. 

MA,  Mathematics,  Ohio  State  University,  1948. 

PhD,  Mathematics,  Ohio  State  University,  1953. 

Dr.  Moranda,  now  a  Senior  Information  Systems  Advisor  at  MDAC-HB, 
was  the  principal  investigator  of  a  contract  with  AFOSR  to  investigate  the 
development  of  quantitative  methods  for  software  reliability.  He  is  also 
working  on  software  validation.  During  the  first  9  months  of  his  employment 
at  MDAC  he  was  principal  investigator  for  software  reliability  IRAD.  He 
developed  mathematical  models  for  software  discrepancies  and  applied  them 
to  failure  date  to  obtain  estimates  of  the  error  content  of  software  packages 
and  estimates  of  their  period  of  error-free  performance.  For  3  years  sub¬ 
sequent  to  that  assignment,  he  analyzed  and  developed  logistics  model  for  the 
YC-15  STOL  aircraft.  He  is  presently  assigned  to  the  Low  Altitude  Defense 
System. 

Prior  to  joining  MDAC  in  1971,  Dr.  Moranda  was  Manager  of  Systems 
Analysis  at  Computer  Real  Time  Systems,  Newport  Beach,  was  employed  by 
North  American  Rockwell  for  6  years,  and  was  technical  advisor  to  the 
director  of  data  management  systems  at  Autonetics  Information  Systems 
Division.  He  participated  in  several  systems  studies  and  development 
efforts.  In  the  field  of  transportation,  he  was  responsible  for  developing  a 
system  framework  for  the  analysis  of  advanced  marine  transportation  sys¬ 
tems.  Additionally,  he  participated  in  a  quantitative  analysis  of  the  opera¬ 
tions  of  the  over-the-counter  trading  department  of  Merrill,  Lynch,  Pierce, 
Fenner,  and  Smith;  an  overview  study  of  the  American  Stock  Exchange;  and  a 
systems  analysis  of  the  U.S.  Federal  Court  System. 

In  1967  he  was  appointed  scientific  advisor  to  the  director  of  manage¬ 
ment  systems,  in  which  capacity  he  developed  methods  of  economic  fore¬ 
casting  of  sales  and  other  business  parameters  employing  random  wavelet 
concepts.  This  work  led  to  the  formulation  of  a  simulation  model  which  pre¬ 
dicted,  in  a  balanced  way,  the  sales,  profit,  cash  flow,  headcount,  backlog, 
and  facilities  requirements. 
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On  the  California  Integrated  Transportation  Study,  he  performed  system 
synthesis,  tradeoff  studies,  and  mathematical  analyses.  In  recent  years, 
he  presented  five  lectures  in  the  transportation  field  at  leading  universities: 
October  1966,  "Advanced  Concepts  in  Transportation  Planning,"  Carnegie 
Institute  of  Technology;  June  1966  and  June  of  1967,  "Application  of  Systems 
Analysis  to  Large  Scale  Systems  -  Transportation,"  University  of  California 
at  Los  Angeles;  Spring  1967,  organized  and  administered  a  full-time  upper- 
division  course  in  Dynamic  Modeling  for  the  University  of  California  at 
Berkeley,  in  which  he  also  delivered  two  lectures,  including  a  summary  of 
the  California  Transportation  Study  and  the  application  of  analytical  techniques 
to  the  study  of  transportation  problems.  . 

Prior  to  joining  Autonetics,  Dr.  Moranda  was  at  the  Aeronutronics 
Division  of  the  Philco  Ford  Company,  engaged  in  operations  analysis.  On  a 
special  assignment  to  the  Ford  Motor  Company,  he  was  responsible  for 
mathematical  modeling  of  the  complete  automobile  production  process.  Other 
assignments  included  development  of  a  methodology  for  handling  fragmentary 
and  unreliable  data  in  a  damage  assessment  center  and  development  of  a  war 
game  model  for  assessing  military  missile  effectiveness.  He  held  the  position 
of  Manager  of  Systems  Analysis  for  3  years. 
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Section  5 
INTERACTIONS 


1.  P.  B.  Moranda  attended  the  2nd  Minnowbrook  Workshop  on  Software 
Performance  Evaluation,  sponsored  by  Syracuse  University  and  Rome 
Air  Development  Center.  Participated  in  paneL  discussions  on  software 
modeling  and  metrics. 

2.  P.  B.  Moranda  presented  a  paper,  "Error  Detection  Models  for 
Application  During  Program  Development,"  to  the  National  Computer 
Conference,  Anaheim,  California,  on  22  May  1980. 

3.  P.  B.  Moranda  presented  the  same  paper  at  ACM's  Pathways  to  System 
Integrity,  Gaithersburg,  Maryland,  on  19  June  1980  (paper  published  in 
the  Proceedings  of  the  Conference). 
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j 

2 

2 

6 

19. 

3 

3 

3 

9 

in 

«•  '■  • 

2 

2 

6 

21. 

i 

3 

3 

9 

22. 

> 

2 

2 

6 

??. 

3 

3 

3 

9 

'Program 

60.77 

Pc 

80.77 

Pc 

80.77 

Pc 

39.45 

Pc 

MfclM  Sejaent  Reference 

1.  CO-1!) 

7.  cii/5-ii) 

3.  ni-121 

-  HI  C$*MUNt?DP  for  MODULE  'MAIN  '  - 


TRIAMG 


* 


« 


* 


1.  CO-1) 

2.  C  3-4, J4 ) 

1.  C 3 4-17 ) 

4.  C 37/  2-3) 

5.  C37-1«] 

f.  C31/3&-37) 

7.  C3/0-3) 

P.  Cl-il) 

9.  ril/7-3) 

10.  r l 1-13) 

11.  C 1 3 - 1 8  ) 

12.  C11-11/J4) 

n.  ri8/2o-?2) 

14.  C22/20-22) 

lr..  C2*>-?7) 

If.  C  27-23/ 32-3 J) 
17.  931/21-77) 

1  *».  CJ3-U) 

1».  r3 7/21-31} 

20.  CJl/2j-31) 

?t.  r 31-33) 

27.  ri 3/13-13) 

?■».  C  3/10-11) 


Sequent  Reference 


S3  C4«"3KTT0?  for  ”7  Jli 


T^I ASG 


Cas* 


7  C 


MAIN 


TRUNK 


1. 

2. 

3. 


1. 

2. 

3. 

4. 

5. 

6. 
7. 
-9. 
9. 

10. 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 

20. 
21. 
27. 


66.67  Pc 


1 

1 

0 


86.96  PC 


1 

1 

0 

2 

1 

3 

2 

1 
1 

2 
1 
0 
2 
1 
2 
0 
1 
2 
3 
•> 


23. 


3 

2 

3 


as*  1 

33.33  Pc 


0 

1 

0 


78.26  Pc 


1 

1 

0 

2 

1 

3 

2 

0 

1 

2 

0 

0 

2 

1 

2 

0 

1 

2 

3 

2 

3 

2 

3 


33.33  Pc 


0 

0 

1 


86.96  Pc 


1 

1 

0 

2 

1 

3 

2 

3 

1 

2 

1 

0 

2 

1 

2 

0 

1 

2 

3 

2 

3 

2 

3 


Suaaary 
100.00  Pc 


1 

2 

1 


d 


96.96  Pc 


<OI7'CO'>6i?UOC'U)OiOmO'U)»<J''CUOOu)U' 


UIN 


1.  CO-11) 

7.  Cll,5-ll) 
3.  £11-121 


Segment  Reference 


mo  i 


-  13  CSSMOtilTOR  for  MQOULP:  'MAIN 


TRIADS 


* 


1.  CO-3) 

2.  C3-4,34) 

3.  C34-37 ) 

4.  C  37/2-3  ) 

5.  C 37-333 

6.  C34/36-37) 

7.  C  3*5-3) 

8-  rg-ll) 

9.  Cll/7-3) 

10,  ril-13) 

11.  C13-13) 

17.  Cie-19,34) 

13.  C 18/20-22) 

14.  C2?,70-22) 

15.  C  27-27) 

16.  C27-29, 32-33) 

17.  C33/73-27) 

18.  C33-3  4) 
l?.  C  27#  2>-31 ) 

20.  C  31#29-3l  ) 

21.  C31-3 3) 

27.  C 13* 15-19  ) 

23.  C  9# 1 0-11 ) 

MO  C$$H0!»1T0H  for  MODULE 


'TRIAMG 


Segment  Reference 


A-fl 


Trial  Statistics 


•lumber  of  trials(,f)=  9 

Value  of  Xit 

XC  13  :  1 

XC  ?i  :  1 

xc  m  :  i 

xc  43  :  4 

Humber  of  Xi{N)=  1 


d 


MC0OIVMU 


-  ‘ilrVi  llUm  I 


A-10 


Appendix  B 

OUTPUT  FROM  A  CONSTRUCTED  CASE 
AND 

CONSTRUCTED  CASES  LISTINGS 


OUTPUT  FROM  A  CONSTRUCTED  CASE 

Case  1  Summary 


«UIM  56.67  Pc  66.67  Pc 

t.  1  1 

2.  0  0 

3.  :  l 

TRIHM7  73.  U  °c  73.  <>1  Pc 

1.  1  1 

2.  1  1 

3.  1  1 

4.  >  2 

•J.  !  1 

6.  i  3 

7.2  2 

8.  2  2 

r*.  1  1 

10.  1  2 

11.  u  0 

12.  1  1 

n.  i  l 

14.  J  0 

U.  i  1 

16.  I  l 

17.  0  0 

13.  1  1 

19.  )  0 

?r.  j  o 

71.  J  0 

22.  3  2 

23.  1  3 

*Proqrar*  72. 3y  Pc  77.03  Pc 


r-  V  «<-•**  -yr 


-3 


Segment  Reference 

1.  CO-10) 

*  2.  rlO/5-10) 

3.  UO-UJ 

-  Ml  C$$“3MIT1»  for  MOlHJLF  'MAIM  '  - 


;  Segment  °»Fereiice 


1. 

ro-3i 

2. 

C3-4,  3c.) 

3. 

T3ft-3j) 

4. 

C 79, 2-3) 

:>• 

r  J  J-  )  3  7 

6. 

C35,  V3-39) 

7. 

T3,5- j) 

3. 

L‘l-11) 

O. 

0  11,7-3) 

1C. 

T 11-11) 

* 

11. 

C  1 3-1  7) 

12. 

Cl  7- 29, 16) 

13. 

Cl  9,31-23) 

* 

1*. 

C23,?i-'*7) 

U>. 

C  23-2  >) 

1^. 

020-39, 34-35) 

* 

17. 

C3j,  ?-*-:» )) 

10. 

C3G-^. ) 

* 

19. 

CJ9,  Jt-37) 

* 

?C. 

0  3  3,  31  -  3  3) 

* 

21. 

*■  J.3-1  j  ) 

??. 

C 1 7, IS- I 9 ) 

23. 

0,13-11) 

-  Monitor  I'ceJicat*  - 
Pr«e  tc  ate 

1.  .'r 

2.  MMT 

3.  TVJM 


Type  Minimum  "-v<i  pun 

Intejer  t 

Real  .909000000009000 90  .  t  no OlOOOl 090000FV 

Peal  .900000000900900  £«-9  9  .03  )OOOOOCOOOOOO ';♦! 


B-3 


Trial  Statistic* 

Muster  of  trial=s(?)=  1 
Value  of  Vi: 


Nunoer  of  Xi  (N)  =  0 


k 


8-4 


CONSTRUCTED  CASES  LISTINGS 


a-  i 


2 

3 

4 

5 
5 
7 

3 

3-  10 

11 

l? 

13 

14 


PROGRAM  VAIN 
IMPLICIT  INTEGER!  4-?.) 

C 

INTEGER  1*0) 

PEAL  M4,3),R 

C  , 

DATA  }/0*0/4»0#3»0/5»0/2«0/4«3/0«0/7«l/)»0|r3,0/ll*0/12»0 

r 

UI,rv(UNIT  =  2!0/OEVICE='IjSK:'/FTLE='aPAD.RE3',ACCirSS  =  'Sr()aUT')  i 

r*  ■ 

N  =  3  j 

N  i  =  4  ’I 

M=1  j 

C  ) 

00  10  K=  1/ M  | 

yRim20,10l)K,<(A<l,.J),J-l,N),I  =  l,\M) 

CALL  TRIANG(TP,A,N) 

VRITE(20,102)F,((A{I,J)/J=1,N),I=1,LF) 

10  CONTINUE 

3  'O? 

c  ; 

101  B-IRSATO  BEFORE  TRI ANGULAR I ZATION  (*,  II ,  '  )  *7/ jOX,  P20.  10  >  )  ■ 

10*  rjn».kTC  AFTER  TRI A NGULARIZATION  C  % II ,  *  >  '//3f  3X,  *20. 10))  j 

c 


( 


n 


S«maUTIWE  TRrANG(IP,A,J<) 
ILLICIT  INTECER(A-Z) 


I7TKG2?  IP(3  ) 

A( 4/3J/7 
okAL  w3«T 


0- 

2 

3- 

5 

5 


rr(v)=i 

P)  6  K  =  l,<\' 

TF(K.SQ.N)  GOTO  5 

m=K+i 

'•'sir 

DO  1  I=KPl#* 


ft- 

9 

I F (  Aft  S  (  *  (  I  /  K  )  )  •  C 

10- 

11 

1 

CTMTINUE. 

12 

!P(<)=V 

13- 

14 

TFfM.NE 

.*r>  iF(*o=-rp(N) 

15 

7=A(’4,*) 

15 

A(K,K> 

17 

'(*/K)= 

T 

13 

•*'0«T=ABS(T) 

c$$<njJTTn 

>  =  «!•'  ALC'PNT); 

n- 

29 

.0)  G3T>)  5 

21 

no  2  i  = 

fPl/F 

22- 

23 

2 

24 

pn  4  J- 

KP1,* 

25 

T  =  A<  «,,J) 

2fi 

A<^,  1) =t (* / J ) 

27 

A(',J)=T 

2ft 

TMC'i«=A3S(T) 

‘  i’!f  n 

■’-3?-  AT.(TMni); 

29- 

30 

TF(T.£9.0>  GOTO 

J1 

DO  3  I =.KP1  / *! 

32- 

33 

a ( r / j) = 

3  4- 

?5 

t 

*  CJSi’TlNUE 

36- 

33 

j 

If (A(K, 

■  K).EJ.O)  !P(N')=0 

3ft- 

39 

3  C 

'"ITIf.Uf 

43 

n 

T  J  °  V 

41 

1? 

jf; 

t! 


B-6 


IP 

%*■ 


